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Background 


George  Mason  University  Systems  Engineering  project 

-  Could  the  integration  of  sensors  address  DoD  surveillance 
capability  gaps  that  currently  exist  in  the  continental  US  (CONUS)? 

-  Inspired  by  Military.com  article,  “DoD  Finds  Cruise  Missile  Defense 
Gaps,”  17  Aug.  2006 

NSWCDD  engineering  team 

Agent-based  modeling  employed  as  part  of  the  early 
systems  engineering  process 

Scope  of  analysis  constrained  to  Northeast  US 
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Current  Situation 


Long  Range  Air  Route  Surveillance  Radars  (ARSR)  and  ATCBI  (IFF) 


ARTCC 


NMCC 


NORAD 


■>  Request  for  Assistance 
Engagement  Orders 


ARTCC  -  Air  Route  Traffic  Control  Center 
NORAD  -  Northern  American  Air  Defense  Command 
NEADS  -  Northeastern  Air  Defense  Sector 
WADS  -  Western  Air  Defense  Sector 


Joint  Surveillance  System  (JSS) 


CONR 

Command 


o 


Sector  Air  Defense 
Tactical  Command 


NMCC  -  National  Military  Command  Center 
CONR  -  Continental  US  (CONUS)  NORAD  Region 
SEADS  -  Southeastern  Air  Defense  Sector 


^3 

Air  Defense 
Aircraft 
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Homeland  Sensor  Network  (HSN) 

System  Options 


Air  Route  Surveillance  Radars  (ARSR)  and  ATCBI  (IFF) 


Joint  Surveillance  System  (JSS) 


Air  Defense 
Aircraft 


10/17/2007 


ARTCC 

Multifunction  Phased 
Array  Radar  (MPAR) 

r  '“K 


NMCC 


CONR 

Command 


NORAD 


Airport  Surveillance  Radars  (ASR) 


Sector  Air  Defense 
Tactical  Command 


Tethered  Aerostat 
Radar  System  (TARS) 
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Systems  Integration  Options 

(DC  Capital  Region  IOC) 

Do  Nothing.  Live  with  Standard  Operationai 
Procedure  (SOP)  to  fiy  Airborne  Eariy  Warning 
Aircraft  (e.g.  AWACS)  and/or  depioy  Sea- 
based  sensors  (e.g.  Aegis  ships)  for  increased 
sensor  capabiiity 

integrate  Airport  Surveiiiance  Radars  (ASRs) 

Procure  and  integrate  a  Phased  Array  Sensor 
at  a  miiitary  faciiity  ciose  to  DC  Capitai  Region 

Procure  and  integrate  Tethered  Aerostat  Radar 
Systems  (TARS)  at  miiitary  faciiities  ciose  to 
the  DC  Capitai  Region 

Combinations  of  the  above  options 
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Engineering  Process 


Defined  metrics; 

-  Value  Hierarchy 

-  Key  Performance  Parameters 

-  Measures  of  Effectiveness  and  Performance  (MOEs/MOPs) 
Developed  alternative  system  architectures 
Developed  agent-based  model  architecture 
Refined  model  details 

Ran  model  once  matured 
Performed  statistical  analysis 
Estimated  cost,  maintainability  and  reliability 
Drew  conclusions 
Made  recommendations 


I 
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HSN  Value  Hierarchy 
&  Agent-Based  Modeling 


Agent-Based  Modeling  Used 


Homeland  Sensor 
.Network  Value  Hierarch^ 


Supportability  (10%) 


Duals  (20%) 


Continuous  Track  (20%) 


Shared  (20%) 


) 


< 

C 


Availability  (80%) 


Maintainability  (20%) 


) 


ID  (60%) 


Position  (40%) 


Cost  was  evaluated  as  an  independent  factor 
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Agent-Based  Modeling  Platform 


-  Multi-agent  programming  language 

-  Integrated  modeling  environment 

-  Audience;  teaching  and  research 

Who  develops  it? 

-  Center  for  Connected  Learning  and  Computer-based  Modeling, 
Northwestern  University 

What  does  it  support? 

-  Rapid  model  development 

-  Statistical  experiments 

What  else  is  important? 

-  Free 

-  Cross-platform  (Java) 

-  Large  and  active  user  community 
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Modeling  and  Analysis  Approach 


Agent-Based  Performance  Model 

*  Control  experiment  factors 

*  Simulated  aircraft  flight  profiles 

*  Simulate  sensor  (IFF  and  radar) 
detections 

*  Model  sensor  detection  errors 

*  Generate  sensor  report  file 

*  Real-time  calculation  of  MOPs; 

-  Coverage 

-  Continuity 


Slide  10 


Data  Analysis  Program 

*  Correlation  of  sensor  reports 
(tracks)  from  multiple  sensors 

-  In  accordance  with  MIL-STD 
6016  (Link  16) 

*  Calculate  FAA  picture 


*  Offline  calculation  of  additional 
MOPs: 

-  Shared 

-  Dual  tracks 

-  ID  accuracy 

-  Position  accuracy 
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HSN  Performance  Model 


sensor-config 

A  -  Baseline 

t| 

Model  Inputs  and  Controls 

Model 


r 


Vi'-'  'HJj;? 


Setup 


S8Ff 


Jft-  > 


file-output 


dump-true-values 


Aircraft  Parameters: 


'M' 

C»p«  w 

Cilv+  gfeiii<-3  ^ 

Stjffoir^  Pr  Dt.A'vWRE  K 

■  ,  OFtOLUMilA*  f ^ 

n  ■  ,  .  j  S. 


;  '. 


percent-flying-routes  0  % 


num-large-comm-AC  100 


"* '  .  \ 


^irii0w7  ^ 

■  rr  X-CR  ^  '  ._  ^  . ’i  ^^  ! 


R^i 

Kw 


num-corp-jets 


r 


num-light-AC 


50 


num-prop-AC 


50 


50 


IFF-failure-rate  10  % 


/ 

M:,. 


raff:  ^  * 

..  &  •"oiB  ;  T 


1  a  r>»Mi 


Model  Inf 


L 


Model  Calculations 


Clock 


117 


7o  A/C  Detected 


98.76 


%  Continuity 
96.59 
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Model  Design 


*  Aircraft  randomly  placed  across  airspace 

*  Sensors  read  from  input  file 

-  Primary  radars 

-  Secondary  radars  (beacons) 

*■  Sensor-to-aircraft  range  checks 

*  Sensing  modeled  using  probability  distributions 

-  Detection 

-  Accuracy  (range,  azimuth,  altitude) 

*  Accuracies  calculated: 

-  Position,  speed,  course 

*  Aircraft  colors  changed  based  on  detection 
*■  Simulation  cycle: 

-  Sense  Aircraft 

-  Move  Aircraft 

-  Move  Sensors 

-  Update  Display 
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Experimental  Factors 


Aircraft  Density 


Aircraft  Behavior 


Aircraft  Types: 

•  Large  Commercial 

•  Corporate  Jets 

•  Private  Prop  A/C 

•  Light  A/C 


Beacon  (IFF) 
Failures 


Baseline 


System  Integration 
Alternatives 


Air  Route  Traffic  ^ 
Control  Center 
(ARTCC) 


Measures  of 
Effectiveness 


Supplemental  Sensors 

•  AWACS 

•  Aegis  Ship 
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System  Configurations  Analyzed 

^  ■■ 


Sensors  Integrated 

Name 

ARSR-4 

ASR-9 

Aerostat 

(TARS) 

Land-Based 
Phased  Array 
Radar 

AWACS 

Aegis 

Ship 

Radar 

Track  Correlation 
Capability 

Baseline 

X 

JSS  (Uncorrelated) 

Baseline  +  SOP 

X 

X 

X 

JSS  supplemented  w/ 
correlated  data 

ASR  Integration 

X 

X 

JSS  supplemented  w/ 
correlated  data 

ASR  +  SOP 

X 

X 

X 

X 

JSS  supplemented  w/ 
correlated  data 

Aerostat  Integration 

X 

X 

All  sensor  data 
correlated 

Land-Based  Phased 
Array 

X 

X 

All  sensor  data 
correlated 

ASR  +  Aerostat 

X 

X 

X 

All  sensor  data 
correlated 

ASR  +  Phased  Array 

X 

X 

X 

All  sensor  data 
correlated 

ASR  +  Phased  Array  + 
Aerostat 

X 

X 

X 

X 

All  sensor  data 
correlated 

All  Sensors 

X 

X 

X 

X 

X 

X 

All  sensor  data 
correlated 
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Decision  Analysis  Tree 


All  values  normalized  between  0  and  1 
Results  that  did  not  meet  threshold  KPP 
values  were  assessed  at  0. 


8ASR/MxleS  a301| 


a907 


.  Irtecration 

Alternative 

a336 

.  DDNottingj _ 

8ASR+2TARSI 


8ASR+IVPAR 


8ASR+2TARS 
+  IVPAR 


0.8891 


a7D3| 


0.907 


a688| 


0.9071 


^rrpl^T^j. 


70.0%, 


0.95 


40.0% 


0.95 

20.0% 


0.92 


◄ 


Gorrpletenss  MOPs 


CdrtinuousTrackI 


20.0% 


0.96 

20.0% 


0.96 


AvalaUlityj _ 80.0%^ 


a^pottebli^ 


110.0%^ 
0.80^ 


Sifjpottability  MOPs 


_  IVHntainablity 


20.0%^ 

◄ 


0.09 

60.0% 


20.0%i 


AcoiacyMOPs 


0.81 


Rasitionl 


0.92 


40.0% 


0.65 


Two  Options  Meet  Objectives  Prior  to  Consideration  for  Cost 


Slide  15 


1011712007 


Normalized  Scores  for 
Measures  of  Effectiveness 


Integration  Options 

Completeness 

(NV) 

Accuracy 

(NV) 

Suppo  liability 
(NV) 

Alternative 

Results 

Cost 

Baseline 

0.24 

0.33 

1.00 

0.336 

$0 

8  ASR ;  Mode  S 

0.19 

0.40 

0.85 

0.301 

$1.0M 

2  TARS 

0.98 

0.55 

0.95 

0.889 

$8.9M 

MPAR 

0.64 

0.79 

0.98 

0.703 

$12.4M 

8  ASR  +  2  TARS 

0.95 

0.81 

0.80 

Cl^907 

$9j^ 

8  ASR  +  MPAR 

0.64 

0.80 

0.84 

0.688 

$12.7M 

8  ASR +  2  TARS  + 
MPAR 

0.96 

0.78 

0.78 

0.907 

$21.0M 

NV  -  Normalized  value  from  0.000  to  1.000 


Alternative  selected  based  upon  Decision  Analysis 
and  Cost  as  an  independent  factor 
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Analysis  Conclusions  and 
_ Recommendations 

Could  the  integration  of  sensors  address  DoD  surveillance 
capability  gaps  that  currently  exist  in  the  continental  US 
(CONUS)?  (YES) 

-  Do  nothing  option  does  not  meet  KPP  requirements 

-  Integration  of  the  JSS  with  2  TARS  and  8  ASR  sensors  provides  the 
best  capability  to  meet  KPPs  at  a  reasonable  cost 

Proceed  with  next  Phase  of  Homeland  Sensor  Network 
(HSN)  development 

-  However,  a  detailed  performance  assessment  with  classified  data 
and  a  higher  fidelity  model  is  recommended  at  the  beginning  of  the 
next  phase 
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Agent-Based  Modeling  (ABM) 
Lessons  Learned 


General 

*  Worked  well  for  the  problem 

*  Was  valuable  as  part  of  an  overall  engineering  process 

-  Early  assessment  of  critical  performance  factors  on  a  large, 
distributed  system  problem 

-  Required  system  architectures  and  requirements  as  inputs  to 
model  design 

*  Forced  detailed  review  of  Measures  of  Effectiveness 

*  Forced  increased  understanding  of  system  alternatives 

NetLoao  Modeling  Platform 

*  Supported  rapid  model  development  and  changes 

*  Provided  direct  support  for  experimental  factors 

*  Required  off-line  data  analysis 

*  Would  not  support  detailed  radar  performance  analysis 
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Questions  ? 


,.-V 

s. 

\ 

% 

\ 

/' 

:f 

'<  /i*' 

V 
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Recent  Pentagon  Assessment 


“Pentagon  assessment  of  U.S.  capability  to  defend  the 
homeland  ...  has  found  what  it  calls  ‘capability  gaps’...” 

-  Military. com  (August  06) 

No  common  operating  picture 

Insufficient  surveillance  coverage  of  NORAD’s  area  of  operations 

Inability  to  detect  small,  low-speed,  low-altitude  targets 

Inability  to  automatically  fuze  information  from  Wide  Area  Air 
Surveillance  family  of  systems 

Air  defense  sensors’  inability  to  reliably  provide  adequate 
tracking  information 

Air  defense  sensors’  inability  to  determine  the  intent  of  an  aircraft 
with  100%  reliability 

Inability  of  sensor  system  to  provide  enough  information  for 
military  officials  to  make  “engagement  decision 
recommendation” 

Inadequate  supply  of  information  to  NORAD  analysts  from  other 
government  agencies 

□  Not  enough  weapons-delivery  platforms  available  to  cover  North 
American  continent 
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Scope  of  Analysis 


Limited  to  Air  Traffic  Control  Areas: 


-  ZDC  (Washington) 

-  ZOB  (Cleveland 

-  ZNY  (New  York) 

Limited  to  States 

-  Virginia 

-  Pennsylvania 

-  New  York 

-  West  Virginia 

-  District  of  Columbia 

-  Maryland 
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How  Aircraft  are  Tracked  Today 

(IFF  Example  -  Primary  Means) 


Once  a  plane  takes  off,  your  pilot  activates  a  transponder  device  that  detects  incoming  radar  signals 
and  broadcasts  an  amplified,  encoded  radio  signal  in  the  direction  of  the  detected  radar  wave.  The 
transponder  signal  provides  the  controller  with  your  aircraft's  flight  number,  altitude,  airspeed  and 
destination.  A  blip  representing  the  airplane  appears  on  the  controller's  radar  screen  with  this 
information  beside  it.  The  controller  can  now  follow  your  plane. 


Air  Route  Traffic 
Ccntrol  Center 
(ARTCC) 
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HSN  Key  Performance  Parameters  (KPPs) 
Originating  Requirements  Document 


MOE 

MOP 

THRESHOLD 

OBJECTIVE 

(1.0)  Completeness 

(1.1)  Coverage:  The  HSN  shall  support  tracking  of 
all  objects. 

95% 

98% 

(1.2)  Duals:  The  HSN  shall  provide  less  then  duals. 

5% 

3% 

(1.3)  Continuous  Track:  For  each  aircraft  tracked,  the 
HSN  shall  hold  contact  of  the  time. 

95% 

98% 

(1.4)  Shared:  The  HSN  shall  provide  a  common 
(shared)  track  picture  among  critical  military  National, 
Operational,  and  Tactical  Command  and  Control  (C2) 
nodes. 

95% 

98% 

(2.0) 

Supportability 

(2.1)  Availability:  The  HSN  shall  have  an  availability  of 

■ 

99.999% 

99.999% 

(2.2)  Maintainability:  The  HSN  shall  be  maintainable  by 

2  operators  with  a  mean  time  to  repair  of  _  hours. 

8 

4 

(3.0) 

Accuracy 

(3.1)  ID:  Track  identification  shall  be  percent 
accurate. 

90 

95 

(3.2)  Position:  HSN  track  position  accuracy  shall  be 
accurate  within  _  nautical  miles. 

5 

3 
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Decision  Analysis  Approach 


Completeness  and  Accuracy  MOE  results  derived 
from  performance  modeling  and  data  analysis 
program. 

Supportability  MOE  approach: 

-  Maintainability:  based  upon  number  of  site  for  each 
design  solution  -  i.e.  more  sites  equates  to  more  time 
required  for  troubleshooting  and  repairs. 

-  Availability:  calculation  based  upon  system 
redundancy  of  each  solution. 

Availabili  ty  =  [  1  -  ( 1  -  ^  “  ] 


^sensor "  (design)  specified  availability  of  each  sensor  (e.g.  TARS  =  0.98) 

Nsensor  -  number  of  overlapping  sensors  (e.g.  2  TARS  with  overlapping  coverage) 


I 
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